Interactive Multi-objective Reinforcement Learning in Multi-armed Bandits with Gaussian Process Utility Models

In interactive multi-objective reinforcement learning (MORL), an agent has to simultaneously learn about the environment and preferences of user, in order quickly zoom on those decisions that are likely be preferred by user. this paper we study MORL context multi-armed bandits. Contrary earlier approaches force utility user expressed as a weighted sum values for each objective, do not make such stringent priori assumptions. Specifically, only allow non-linear preferences, but also obviate need specify exact model class function must fall. To achieve this, propose new approach called Gaussian-process Utility Thompson Sampling (GUTS). GUTS employs parameterless Bayesian any type function, exploits monotonicity information, limits number queries posed ensuring questions statistically significant. We show empirically can regret highly sub-linear arm pulls. (A preliminary version work was presented at ALA workshop 2018 []).